s 



.T75 



Thirteenth Series, No. 12 May 27, 1922 

S^arlj^ra Ololbg? lulblm 




Supplementing the Hillegas Scale 

A Description of the Derivation and Use of the Nassau 
County Supplement to the Hillegas Composition Scale 



By M. R. TRABUE, Ph.D. 

PROFESSOR OF EDUCATION, UNIVERSITY OF NORTH CAROLINA 



Published by 

3(rart|ar0 (Ball^B^. (Halambm llnitirrBUg 

525 Weet 1 20th Street 
New York Qty 



M« 



"oe'-aaft 



Qlrarljprs CdaUrgf ?inlUtin 

Thirteenth Series, No. 12 May 27, 1922 

Published twelve times a year: Fortnightly in September, October, and May, and 

monthly from November to April, inclusive. Entered as second class matter January 

15, 1910, at the Post Office at New York, New York, under the Act of August 24, 1912. 
Acceptance for mailing at special rate of postage provided for in Section 1103, 

Act of October 3, 191 7, authorized. 



Supplementing the Hillegas Scale 

A Description of the Derivation and Use of the Nassau 
County Supplement to the Hillegas Composition Scale 



By M. R. TRABUE, Ph.D. 

PROFESSOR OF EDUCATION, UNIVERSITY OF NORTH CAROLINA 



reprinted from 

Teachers College Record, Vol. i8, No. i, January 1917 

revised edition 



Published by 

STparlirrH (EoUrgp, ©nlumbia llniuprHity 

525 West 1 20th Street 
New York City 






OCT 23 '22 

^C1A683863 



^0 I 



SUPPLEMENTING THE HILLEGAS SCALE* 

During the spring session of 1916, some of the advanced stu- 
dents in the department of educational administration at Teach- 
ers College were engaged in an educational survey of Nassau 
County, Long Island. > In the course of this survey the educa- 
tional achievements of public school pupils were measured by 
means of standard educational scales and tests. The measure- 
ments of the quality of English compositions written in Nassau 
County suggested very strongly the need for two supplements 
to the Hillegas Scale.' In the first place, there seemed to be 
a distinct need for a supplementary scale composed of composi- 
tions of the same general type as those written by Nassau County 
pupils, and, in the second place, it appeared desirable to set up 
some tentative standards indicating the quality of English 
compositions to be expected from the pupils of any given school 
grade. The discussion which follows is a report of certain efforts 
to supply these two supplements to the Hillegas Scale. 

In spite of all the criticisms of and objections to the Hillegas 
Scale, the fact remains that it is one of the most useful measur- 
ing instruments in the whole field of education. The objective 
character of the scale has been extremely useful. By means of 
this scale it is now possible for one person to understand and 
to identify the quality of written English indicated by another 
person. Intelligent use of the Hillegas Scale by teachers and 
supervisors tends to place emphasis on the quality of the written 
composition rather than upon the personal relations of the 
writer and the teacher. Any supervisor who employs the scale 



*Copyright, 19 17, by Teachers College. 

^ Trabue, M. R., Report of a Survey of Public Education in Nassau County, 
New York. Albany: University of the State of New York, Bulletin No. 652, 
December i, 191 7. 

' Hillegas, M. B., A Scale for the Measurement of Quality in English Composition. 
Published by the Bureau of Publications, Teachers College, New York City. 



2 TEACHERS COLLEGE BULLETIN 

with a group of his English teachers, and who then takes the 
trouble to discuss with such teachers in the light of this experi- 
ence just what constitutes general merit in English composi- 
tion writing, will be able to show as a result a much greater 
agreement of opinions and of school marks in this subject. In 
other words, the use of the Hillegas Scale calls attention to and 
defines fairly well what we mean by general merit in English 
composition. 

One of the defects which is quite frequently pointed out by 
those who object to the Hillegas Scale is the fact that the first 
three samples of this scale are not the compositions of children, 
but are artificially constructed samples. To some of the other 
samples the objection is made that they are so short as to afford 
very little evidence of their true quality. As an example we 
may point out Sample 534, which represents Quality 5.85 on 
the scale. 

Fluellen 
The passages given show the following characteristic of Fluellen: 
his inclination to brag, his professed knowledge of History, his com- 
plaining character, his great patriotism, pride of his leader, admired 
honesty, revengeful, love of fun and punishment of those who 
deserve it. 

This confused sentence is hardly sufficient to give one a definite 
impression as to its quality. Objection has also been made 
to the fact that the compositions appearing on the scale are not 
all of the same type. Some teachers claim that they find it 
difficult to compare the quality of two such compositions as 
the one shown above and the one which follows it in the Hille- 
gas Scale (Sample 196, Quality 6.75). 

Ichabod Crane 
Ichabod Crane was a schoolmaster in a place called Sleepy Hollow. 
He was tall and slim with broad shoulders, long arms that dangled 
far below his coat sleeves. His feet looked as if they might easily 
have been used for shovels. His nose was long and his entire frame 
was most loosely hung to-gether. 

Some slight additional criticism has been based upon the in- 
equality of the intervals between the values of samples on the 
scale. The interval between the values of the first and second 
samples is, for example, 1.83 units, while the interval between 



SUPPLEMENTING THE HILLEGAS SCALE 3 

the second and third samples is only .77 of a unit, and the inter- 
val between the eighth and ninth samples is only .66 of a unit. 
Many of the objections made to the Hillegas Scale are of very 
little consequence other than that they offer excuses for not 
making use of the scale. It is nevertheless worth while to at- 



NASSAU COUNTY SUPPLEMENT TO THE HILLEGAS SCALE FOR 
MEASURING THE QUALITY OF ENGLISH COMPOSITIONS 

Directions for measuring: Compare the quality of your composition with the quality 
of the samples on the scale. Assign to your composition the numerical value of that 
evaluated sample which most nearly equals it in merit. 



0. 



.004 



1.1 



1.06 



1.9 



1.93 



2.8 



2.81 



3.8 



3.84 



What I should like to do next Saturday 
I went going on to the Dox Saturdaye dnd day we the boys and 
I well going home and I well going the boys, and I will going these 
read in and they to night, and we or night. I well going a ground 
shalt and I gone out I will going to shea shouse and I will shoe or 
the skill of the shea of night. 



I intend to mak a snou man and make an fort and fort snou ball 
at chidern and hau I whist ma frant carolyn cole what were me I 
will going to the mauiss on Saturday. 

Georga will come went me. 

at night I will going out went my mother to the marce 

I will mak the snou man and the fort in the moning and in the 
aftermoon I will go to the mauies. 

I whist there whest school on Saturday 



one next S aturday I expect to go to the city leve next G aturday 
to see my ofriend archie king I am going to grow to the baning 
balys circus with hime next S aturday fefore I go I have to do my 
jobs feedsing the cows ard horse ard chinkens and geese next Saturday 
My friend is a very good fellow to go and see So my mother S iad 
" If I do my work during Easter week vacation I can go to the barning 
baley circus with, hime 



Once a pon a time there was a girl. One day she asked me what 
I was going to do next Saterday so I said, "I am going to go for a 
swim." And she said, "thats 

just where I am going to." next Saterday came we both went 
down together. We came home at noon time, after dinner we 
went to the picktures. There we had a good time. And then came 
home at night. 



I would like to go out in the after noon and play catching the 
ball. Go over to Bertha's house and have a few girls to come with 
me and be on each others side. I have a tennis ball too play with. 
The game is that one person should stand quite aways from another 
person and throw the ball too one then another. Someone has to 
be in the middle and try too get the ball a way from someone then 
she takes this persons place who she caught the ball from. Then 
till every person has a chance. 



TEACHERS COLLEGE BULLETIN 



5.0 



4.97 



7.2 



8.0 



Next Saturday I should like to go away and have a good time on 
a farm. I should like to watch the men plowing the fields and 
panting corn, wheat, and oats and other things planted on farms. 

Next Saturday I will go to the Pioneer meeting if nothing happens 
so that I cannot go. I should like to go swimming but it is not 
warm enough and I would catch a bad cold. I should like to go to 
my aunts and drive the horses, I do not drive without some older 
person with me, so I cannot go very often. 

I should like to see my aunts cat and her kittens, too. I think 
I can, to. 



I should like to join my girl friends, who are :?oing to the city on the 
9:05 A. M. train. They are going shopping in the morning and will 
^Q have lunch to-gether, then they are going to the Hippodrome After 
the Hippodrome, they are all going home to dinner to one of the 
girls houses, she lives on Riverside Drive so they expect to take the 
"Fifth Avenue Bus" up there. The evening will be devoted to 

6 01 P'^ying games, .'dinging and dancing. 

If I had a thousand dollars to spend, I think I would take a trip 
to San Francisco by train with the rest of the family, and stop at 
a sea-side hotel. It would be glorious to see the surf again, and 
to escape from the cold blustering weather of December for the 
balmy breezes of the ocean, and the whiff of orange blossoms. 

We could take long drives under shady trees, visit the orange and 
olive groves and bathe in the surf. Think of bathing in the ocean 
n December! 

Coming home again I should enjoy stopping at Yellow Stone 
Park. It would be lots of fun to camp out, and to ride over the 
prairies on frisky ponies. It would be very interesting to notice 
the change of climate as we got farther east, and to go to bed on 
the train one evening feeling warm, and waking up the next morn- 
ing feeling very chilly. 

I am afraid by the time I would get home a thousand dollars 

7 22 would be pretty well used up; but if not I would like to give a party. 

One Sunday, towards the end of my summer vacation, I was in 
bathing at the Parkway Baths. In the Brighton Beach Motor 
drome, a few rods away, an aviation meet was going on. Several 
times one of the droning machines had gone whirring by over our 
heads, so that when the buzzing exhaust of a flier was heard it did 
not cause very much comment. Soon, however, the white planes 
of "Tom" Sopwith's Wright machine were seen glimmering above 
the grandstand. Everyone stood spellbound as he circled the track 
several times and then headed out to sea. He was seen to have a 
passenger with him. Suddenly, the regular hum of his motor was 
broken by severe pops, and the engine ran slower, missing fire badly. 
In response, to Sopwith's movements, the big flier tilted and swooped 
down to the beach from aloft like an eagle. The terrified crowd 
made a rush to get out of the way as the airship came on, but Sop- 
with could not land on the beach, but skimmed along close to the 
water instead. Suddenly his wing caught the water, and the big 
machine somersaulted and sank beneath the waves. The aviators 
soon came bobbing up and were taken away in a launch, but the 
g 00 accident will not soon be forgotten by those who saw it. 



SUPPLEMENTING THE HILLEGAS SCALE 




The courage of the panting fugitive was not gone; she was game 
to the tip of her high-bred ears; but the fearful pace at which she 
had just been going told on her. Her legs trembled, and her heart 
beat like a trip-hammer. She slowed her speed perforce, but still 
fled industriously up the right bank of the stream. When she had 
gone a couple of miles and the dogs were evidently gaining again, 
she crossed the broad, deep brook, climbed the steep left bank, and 
fled on in the direction of the Mt. Marcy trail. The fording of the 
river threw the hounds off for a time; she knew by their uncertain 
yelping, up and down the opposite bank, that she had a little respite; 
she used it, however, to push on until the baying was faint in her 
ears, and then she dropped exhausted upon the ground. 



Note. — The first seven of the above compositions, values o to 6.0, were 
written during the month of April, 1916, by children in the elementary grades 
of the schools in Nassau County, New York. The last three compositions, 
values 7.2, 8.0 and 9.0, were selected from compositions which have previously 
been published by Professor E. L. Thorndike.^ 

The value assigned to " The Hunted Deer " (9.0) is that given it in the 
Thorndike Extension of the Hillegas Scale.' The value assigned to each of 
the other compositions, values o to 8.0, is in each case the median rating of 
139 judges who employed as the basis of their ratings the Hillegas Scale for 
English Composition by Young People.* 

The unit of quality is the median deviation from the median judgment of 
the group of 202 judges used by Dr. Hillegas in securing the final values of 
the compositions appearing on the Hillegas Scale. In less technical terms, 
the unit of quality is such a difference in quaUty as was recognized by exactly 
75% of the original judges and not recognized by the other 25%. For general 
purposes the bold faced values are accurate enough, although more exact 
values are given in small type.* 



tempt to eliminate in so far as possible some of these real or 
imaginary objections to our educational scales. The supple- 
ment to the Hillegas Scale which is here presented does not 
eliminate all of the objections, but it does reduce some of them 
to such an extent that one may at least hope to find an increas- 
ing number of composition teachers making use of objective 
measurement. 



1 Enghsh Composition — 150 Specimens Arranged for Use in Psychological and 
Educational Experiments by Edward L. Thorndike. 

" Thorndike Extension of the Hillegas Scale for Measuring the Quality of English 
Composition. 

* Scale for the Measurement of QuaUty in English Composition by Young Peo- 
ple, by Milo B. Hillegas. 

* Copies of the above Supplement, in a form convenient for use, may be pur- 
chased in any desired quantity. Price, 8 cents per copy by mail; 5 cents in quan- 
tities, postage extra. Published by Bureau of Publications, Teachers College, 
New York City. 



6 TEACHERS COLLEGE BULLETIN 

This supplement may be found somewhat less objectionable 
than the Hillegas Scale inasmuch as the objections mentioned 
above are not so applicable to the supplement as to the scale 
itself. It contains no artificial samples. Each sample on the 
supplementary scale is as true a representation as can be made 
on the printed page of a child's written composition. ^ Each 
of the samples appearing in the supplement is sufficiently long 
that one may readily obtain a real appreciation of its quality. 
Each of the compositions tends to be of the same general narra- 
tive type, which fact gives one a simple basis for the recogni- 
tion of improvement in quality as one passes upward from qual- 
ity o to quality 9. The first seven compositions are upon the 
same topic, '' What I Should Like to do Next Saturday," which 
is closely related to the type of composition one would use in 
writing letters. This is an added advantage, since letter writing 
is no doubt the form of composition for which public school 
pupils will have the greatest use in their lives outside the class 
room. The intervals between the values of the samples in the 
supplement are slightly less unequal to each other than are the 
intervals found in the original Hillegas Scale itself. The rela- 
tive intervals on the two forms of the scale are represented 
graphically in Fig. i (page 14). 

In preparing the above supplement, the writer made use of 
compositions written by the pubUc school pupils in Nassau 
County, New York. Each composition in a group of fifty-five 
hundred written by these pupils had been rated and assigned 
a value^ on the Hillegas Scale by at least two trained judges, 
and in case the first two independent ratings differed, a third 
judgment had been made by still another judge. From this 
mass of roughly evaluated compositions, twenty-eight samples 
were selected upon the values of which the first two judges had 
agreed. In addition to the requirement that his own judg- 
ment and that of the first two judges should agree as to the 
quality of the compositions chosen, the writer also made it a 
rule not to take more than four compositions rated on the same 
sample of the Hillegas Scale. The purpose of this rule was, of 

^ The sample valued at 9.00, it should be noted, is the only sample not written 
by a school child. 

* In these first ratings no effort was made to assign values intermediate to those 
appearing on the Hillegas Scale. 



SUPPLEMENTING THE HILLEGAS SCALE 7 

course, to make sure that there would be in the list two or three 
samples of each quality on the scale. In order to meet this 
requirement for the upper end of the list, two compositions 
pubhshed and partially evaluated by Professor E. L. Thorndike 
were included in the list of thirty samples finally selected for 
the experiment. 

Each of the thirty compositions was carefully mimeographed, 
being identified merely by the first letters of the name of the 
child who had written the composition. ^ About one hundred 
sets of the thirty compositions were prepared, each set being 
enclosed in an envelope to facilitate the administration of the 
experiment. A sheet of directions was also prepared and mim- 
eographed, asking that the compositions be arranged first of 
all in the order of their merit and then rated carefully on the 
Hillegas Scale. A sample sheet of directions, as it was finally 
filled out* and returned to the writer, is shown on page 12. 

One hundred and thirty-nine different judges were used in 
securing the final ratings given each of the thirty samples. These 
judges were all college graduates who had had experience in 
teaching and administrative or supervisory school work, and 
were at the time of making these judgments members of classes 
in educational administration at Teachers College. One group 
of 65 judges were members of the advanced or practicum class 
studying intensively the problems of educational measurement 
during the summer session of 191 6, while the other group of 
74 judges were members of a similar group during the winter 
session of 191 6-1 7. There is no reason to believe that one of 
these groups was by nature any less intelligent or any less able 
to distinguish merit in English composition than the other. 

The first group of 65 judges were assigned the task of rating 
the thirty mimeographed compositions immediately following 
four hours of special class work, two hours one day and two 
hours the next, in which the derivation of the Hillegas Scale 
had been discussed and three or four sets of composition papers 
had each been rated two or three times on the Hillegas Scale. 



^ Identifying letters were " made up " for the two samples borrowed from 
Thorndike. 

* The judgments recorded on this sample sheet are copied in the first horizontal 
ine of Original Record Sheet No. 2. 



TEACHERS COLLEGE BULLETIN 



OQ "H 



'HOOOO 



lOOOOO 



OOOlrtO 



OOlftOO 



0}f ^J 



n irtrtioo 



Ot^rH^OOl 



irtincoNo 



t^Tf OWN 



OOCOlrtLON 



incoco-tN 



S on 



N COminirt 



NOIMOLO 



mt^f-iTi>m 



tOtOt^MlO 



OWOOOIO 



}S of 



tOooNLn 



OincCMN 



t^O-^OOOi 



o/V 



010 05 00 00 



!r>iN^ior~ 



O100N.-IO1 



^MMCvJCO 



OlOt^-*"-! 



Off »7 



t^OlTjiNCO 



ff.o 97 



.-KNCOMCO 



lOLnoO'^ 

(MMOO'-l'-l 



(OOli-HOLO 






nyj vj 



oioomo 



TflNLONN 



Tf rHiniMOO 



0(7 »// 



^DC^O-*00 



^j "d 



OH vyn 



OOlONtOOO 



to ■'t CO 00 in 



to CI C^ 00 t^ 



00 (CIO SO t^ 



ocot^oco 

COMCMMCM 



[^.-(OOrttO 



«/Y iO 



M3Vi ou 



»d "D 



»a fH 



>M. M3 



>PV »/ 



1,0 OD 



^a of 



oj o-j 



COCOM-"!"* 



CO to C^ LOO 
CO •*'*C0^ 



CM CO CO-* 10 



00 10000 1> 
CM 10 CO 10 in 



LOOJtXDlO 
M' 10 00 CM CO 



t>Ot>t><D 
■"tlOCMCOCO 



TjlTtOTj-O 

lOLO'^lOrJ' 



OCM00Ot> 
LO •*■*;£! CO 



lOt>lO00«3 
10 10 CO 10 10 



lOOCMtOtO 
^CO'-HCM.-l 



a)toiot>o 

■q.TtcO'^CM 



(DOOCMO 
-Jf-^CM^CO 



OOCJICMCJIIO 
LOlOCvlTjiCM 



to 00 00 CM CM 
to-* --1 10 CM 



.-lOtOlOO 
COCOCS1C0.-C 



Tf to-* 00 00 

•* CO CO CO CO 



COOCOtDO 
■"tCMCMCO"* 



OiCMCM-*^ 
COlO-*-*C0 



OCOOOCM 

CO-^CO-*-"^" 



ocoooo 

CO CM CO CO CO 



I>CMOLOtD 
CO CO •^■* CO 



OO-^f-iOlO 

Tj-cotrTno 



tocMTjiioin 



001>10->t.-i 

•* CO rococo 



0000(0 00 
10 CO-* CO-* 



t-C0CMlOt> 
•*•*-* CO CO 



1-itDOOOCM 
10 CO CO CO 10 



m MD 



(DlOlOCM.-( 

iom-<ttDco 



Ot>OOC0 05 
lOlO-*lOi-( 



CM (O (J) ■* CM 
(0-*CO->t-* 



03 H3 



-*t>oa)CM 

CO •q"(0-*(0 



05 --KD 10 00 
LOtO^LOlO 



OOOOCMOO 
■*(D-*1010 



t>oc7i--ic:5 
in-*-* to 10 



^J HD 



OKOLOOO 
101010(0-* 



tOCTllOLOOO 

to -^tn 10 10 



»a m 



(O 000 to 00 
CO to to 10 to 



t>0(Jli-00 

10 tOLO tOLfi 



101000CT5 

Tttototnto 



ns m 



OCM-*t>.-l 
to (0 00(00 



OOC-lOt/50 

t> to to 10 1^ 



tOCMO--ilO 

otntoooo 



OOtOCMCMO 

ooooto 



<"S «3 



OlOOCOrl'.-i 
lOlOOlOO 



lOOOOitO OOOlOOtO 
t-OOLOtO 1 OOtOOOf- 



00 Lo to too 



Vtf «7 



l>-*t>00) 

to to to 00 to 



tocM-*(Oto oo-*cri-*-* 
inootoo totoioooo 



OlCOOtOO 



vf pv 



O LO <J) o o 
0(0(0 l>t> 



to 05 CM 000 

otooioto 



O O O 00 o 



CMOOOCn 

otooooto 



IS »il 



oooo-*io 
00000 



OIOIOOO I tooo-* 
OOOOOtOO OOOOtOOiOO 



«S »H 



tD-*Orf<-* 
000 0)000 



-"tOOi-ltOCM UOOOCM'-i 
OitOOOOO OChOOOO) 



SUPPLEMENTING THE HILLEGAS SCALE 



OOCOOtO —oooo 






Ortt-Ttin 



tOLnoNTf 



CMi-iinooT}' 



LnMoo— if- 



PONOOt^f^ 



OOOOIOCO 



MCOtNlO 



MOiwoo ■<toot^ino) 



mmtooiw 



IM (M -H 



COMTflOOO 



^tOOOOlN 



CT*'«f «0 



MtOO)ON 



omotof~ 



XI ^ lo r~ in 



OOO-^LO 



inoootn 



0)03 05 CO ID 



INi-iMM^ 



OOt^OOOO 



00 -H LO LO (N 

-^NCO.-iCO 



I>LO.-iOOOO 
NCOMi-irH 



M-*iMrqr-( 



^ CO CM 



OOCOIOLOC^ 
riCMCOCO-* 



«3t>C>J!£)C0 



i-'i-iCOM.-i 



C^OOOOIO 

rjcgM— ICO 



■* 00 ^CtTO) 
NCMM-*CO 



MCOCOIOM' 



•-(COCOMCO 



oo^]O)coc^0 

COC^C^-HCO 



CJ —■ "3- PJ CO 



LOOlOOOOCM 
CO CO CO CO ;d 



CO-^PJCMN 



OllOCOOO^D 
—iMCO-hCv) 



NOOCVJOlCO 

in --I CO .-1 ■* 



ocooKOLn 

CO CO CM CO-* 



(Ot~-t^C0t> 
CM CM CO CO CO 



0)0;OCO^O 
CM'^LOCO'^ 



■•s"too<rio 



t^-* 00 coo 

CO'^ ■^LOtO 



COOilO'^O 
O.-1-.^COLO 



•^•^C^CMCO 

10 CO CO -sf'-fl 



Ot^LOrHOl 

coTfcoioin 



incO^CMlD 



CMt^OlOOCJ) 

CO CO CO CO CO 



CM CO CM CO Tr 



cMLnoco-* 



COOOOtlO 
COCOUJCOrj- 



t^O-*^DO 
CO-^UI-^LO 



Ol-HCDCMt^ 



<D0OtD!£)i-l 

cOTfcg-ttco 



incMcoco-^ 






■"f O CM CO 'J' 



O)-^00-*CM 

CO-* CO com 






tOC0U5LO;D 



Ol-*C0tD-* 
COCO-flOLO 



LO-t ->J"*t^ 



Ln-*oo>(N 



CM 1/5 000-^ 
CO-* so COLO 






OlCOCMt^OO 



t^Ot-t^LO 

-* in •* '^ lo 



lOtCUDSOCM 
CO CO CM CO-* 



SOOCOt^^ 

coTfinin ■* 



(CCM05CMO 
in Tp^ln to 



in in so-* to 



t^t^t^inoo 
CM CO-* CO in 



oooocjiin 
■* in to CO to 



t^ CO in in in 



cMin-HOoo 
-* CO to in CO 



in in to to c^ 
-* ■*tointo 



CO) to 0)^00 

-* •* msoto 



OOCMOitOC^ 
C0-*tO'*tO 



CM to CM t^ in 

so-* in in in 



t^CMOiCJlCM 

co-*Lnr)'in 



CM in t^ 00 00 
to into in in 



in •*t^ 00 CO 
CO to in in t^ 



05t~-int^cM 
in -* in in to 



CO r~-* to CO 

Tl<tOtO-*t~ 



— lOCMCOi-i 

-*inc^tDLn 



oot-~t^o ■* 
in-*totoc^ 



to f~ in 00—1 
t- to in in t>- 



00 CO CO 0)00 
■v-^tomin 



CO O) --I c-J to 
to CO c^ to to 



tot^ooinin 
to to to to r~ 



C00-Ht^05 



t^OOOt^CM 

soinin WC-- 



oot^toooo 



oootot^oo 
intot^i>to 



tntot-^CMt-* 
t^ t^ t> c^ t-^ 



C^CSt^CMt^ 

c^int>-c^c-» 



to to to 00 00 



t^OlOOOCM 
-*-*tD00 0O 



ot^ t^inoo 



CMt^inoc^ 

00-*tO t^r- 



CMt^CMCMOi 

t^tot^ooo 



t^OOOOTl-O 



•*0)0ino 



'5'-*CJ)t^00 



OOt>tDt>00 



tOr^OOOt^ 

-*S0tOOltO 



CMCMinO'-i 



CO 00 to co^ 



00 1> in 00 CO 
c^t^t^tooo 



oincM— I— I 

t^I>t^O)0) 



to CO CM CM 00 

tooooot^t> 



oocor^-HCO 

to O) to 00 00 



t> in to 0)00 



OOtOt-'CJiCO 

in in t^ 00 c:^ 



inocooco 

t^O)00t^00 



McocMinin 

l^ 00 O) ■* 00 



OCOCOOO) 

oioomoor^ 



t^ CO CM o o 



Sto « 

""oOjs 

1) I, 

x: 1 
m aj 3 

o'^ c 

in cj w 
c:.2 ^ 

^ Mt« 



"2i: 

<U w O 
> N 

HJ 

V £: j: 
tu M^ 

SS ™ 
.9mW 

S^ '^ 
ECi ij 
o aj a 

" c '" 

" £ t^ 

=" DO 

c!3 ij 

O C3 cu 

rag" 
"«o)- 
•«"Ln 
o ♦J 



c 


•^to 




t« 


3 3 










CO -0=0 




c: 






Bl 


Sm 










o 


k5 




J3 


Ma 




- 


b-o 




1 


js » 


r/l 




t.t>- 


C 


u 


CQtO 


o 








o 


h « 


m 


^ 


<U Q 





ri^ 


u 




o>i 


b 




0.x 


u 



lO 



TEACHERS COLLEGE BULLETIN 



OQ ^U 



Oi-iOTrLO 



OOOi-iTtO OOOOIO OONC>3-H OOIOOO 



o;/ ^j 



i-HinLoooiM 



OOl-^OO O-^NNOi CSlOOt-r^Tl" tDON'-i<£> 



H o}{ 



<woomt^ 



i-nocOfHin 






lOlrt— irf<r 00L0.-1OO r-UDlrtcOCJ 



}S of 



OMlrtMin 






OOOmt-HM 0IN«5-<^0 ^tOONlO 



D/V 



ooinoo 

(N M .-H 



o^ooin 



LnoLOi-HO 



^^DCOCOO 00t^Ot>M OlONOHn 



Off »7 



rtrH coined 



O)0in.-n£> 



ff,0 37 SS 



«/v »</ 



"(7 "H 



^A "d 



CO to 00 00 03 



r-iOOCMtO 



Oi/^ OJ^ 






■-H Tf ,-1 (M CO 



.'/V ■'9 



N^IV OS 



oooojt^rj 

LOIOC^NCO 



»<:/ OJ 



COCOi-iCJ>M< 

Ttin-*i-ico 



»ff .»iir 



OOCSlt^MOO 



t^-* (DID 00 

CO CO ■^rt CO 



«/tI V3 



•*-*coLnTj< 



ooo<DO<r> 



«/v »/ 



Tj'tDCOOOO 

•q- in tt tc T^ 



Oi-it^tDN 
inrf rf COM- 



CD -ooint^ 

■^tDTfCOrC 






I.O 03 



(D05Tt"a'ir5 

Tl'Tj'COtO-* 



^a of 



COt^tDincO 
1/5 LOCO to-* 



.-1,-HOOOOO 

imntocoin 



--ic<ii>ooco 

•* Tt LC t> Tj- 



oo-*oiooai 

lOlD'^LT)-* 



lOi-iintrti-i 

■"tCOlOCOTj" 



oj o-j 



■1(7 V3 



OILOOOOC^ 

■^LOLOLnin 



lOOOCONtD 
■*IOCOOOM< 



t^t^OOl^D 

co^tonn- 



fj O) .-I t^ (J» 

(DinTj-tDCO 



COOtXT)--! 
1/5'^ ■^^D-^ 



.-HOOOONN 

O'S'Lnt^io 



NOOCOO-H 
LO t^ -^ tn LO 



^LOLOCOtD 



N(D-HMCO 

Tj-cgLOTj-LO 



oONoomco 
lo ■* LO T)< -a- 



oj HD 



LO<DLOI>'t 



LOlMOlLOr-l 

LfltDCVUOCO 



.-uMai-HCV) 
CD t^ 'J' CO to 



ioooo^oot~ 
LO COLO -a- CO 



■(^ HO 



OOOOOOOl 
^(DtOOO'* 



lOLOt^l>00 
LOLOtDtDtO 



tXDLOtDCO 



^OOOCOO 
001>-*CO!D 



»ff VJ^ 



■q-LOLOOLO 



00 <M to COLD 
(DLOtDt^LO 



t>t>LOCTit> 

to to to to to 



"s .«;¥ 



03 t^ O) "tf tj) 

oirjc^otD 



a^r^Ln-^io 

t>l>t>00LO 



OCOOOtDCO 
t^OOOOt^t^ 



t^t^tOLO'^ 



ffig n;5 



t> t^r-oico 



tOLOTj'ON 

t^t^oooot^ 



t^i>t>ooc^ 



t>lO.-HLOtO 

ooooooot^ 



t^NMLOO 
t^OOtDl>00 



OOOOLO t- 
l>00LOtOt> 



OOLOtDO 

ootoc^r-tD 



HS "7 



OOMC^CO^ 
t>00<DOO0O 



t3>OOOLOCO 
C^tvC^OOO 



COCOI>.-iO 
OOOC^OIOO 



OOt^OOLO 
OOC^tOt^LO 



vf pv 



OCOO-*LO 
O0 00t-O5t^ 



Ot--Ttt^00 

cooot>oot^ 



00 00LOtDL> 



cgLONt^i-i 

00LD00t>tD 



>S »H 



COOOOOLO 
0Ot>O0tJ)tO 



OOOOLOCO 
OOOOt^t^tJl 



Tf OONCOIO 

oot>ooor~ 



^coot>ai 

00 00 00 00 1> 



lOtOO^LO 
0> 00 05 00 00 



OrjtDCqtD 
OlOl^tOtD 



-JOCO'-ILO 

oot^oit^to 



*S »H 



tOOCOf-N 

t^oooooc^ 



t^LOLOOOl 

t- 00 00 05 to 



f-H'S'LOLOLO 

t^i>O>00i:^ 



SUPPLEMENTING THE HILLEGAS SCALE 



11 





2 
o 

M 

O 
O 

U 

o 

H 

Q 

W 

en 

< 

CO 

< 

i-l 

w 


0(7 «y 


MWOO-* 


OOIOCON 

i-H 1 I 


-^OOCDO 


OOOON 


o-*ooo 


^oo-o 


OOOO 




OJV -fj/ 


NNioirsLn 


iOOTi't^Ln 


COIONNCO 


NlOrHt>00 


OtOLOt^OO 


10OCML0 05 

coco rH 


OOtO^M 




y 0^/ 


■HMCMOOO 

rH 1 rHCO 


COrtOlMOO 


CTiMrHW* 


■^OOOrHLO 


C^JLOOOt^OS 
rtCO 


^Nt-tNt^ 


Ot-O^ 




fS of 


O WMO5 00 




lO-*COOO 


COtOrtOOO 


t>OtOCMLO 
•-ICSI <N 


tOlOOt-O 
^<N 


lOO5 00rt 




opV 


000-*OLO 


OO'l'C^lOOC^ 


OOOlOCOLO 


^cqOMOi 


totot>ot> 


'* 0)05'* 00 


to 00 to to 




og »7 


(Mr-i com 


tnoiLOf<iO 
cO'Hcocorg 


TftOOOOCM 


OOtOOONlO 

rHcsi^pgcM 


COOCO-^M 


c<IOooo^^ 

CMCMrHrHC<l 


lO CM to CO 
CM-HCM 




ff.0»7 


ooocsiinoo 


UDLOLntDM 
COCg (NW 


00t0Ot>L0 

■-ICOCJrH.-l 


LOOOCOOIN 


^COPJOOO 

f^cor-irgco 


OtOLOtO* 


OtOO)00 

rHrtCM 




«/V "c/ 


un'-iffiMoo 


CO CO CO CO cj 


001010 05 00 


OI>lO-HO 
CMNlNcgcO 


to 00 to CO 00 
M ^C^4rt 


O00tOO5 1> 

corqrtr^cM 


OCMOOt- 
COCMCM 




Off 0// 




lOOOOOtO^ 

rHCJCOCOIN 


tocot^ot^ 

IM-*CSIC^1^ 


t^ 00 LOCO 00 

rtcortc^csi 


CM-^C^LOO) 
lOCO-HMrt 


cMtoor-io 
*CMrococM 


ooot- 

r-H'*CM 




'^«5 




i-HOlMl>00 


OOOOOON 

rqrtcv)ooc<i 


to to o o to 
cQcoc^c^cg 


■*005<010 

iNCOrtNrH 


Tf tOlNt>tO 
'*'*'* CM CM 


cMtortin 

CO CM CO rH 




on ofv 


i>ooot^m 
cocj.-iTj'co 


0)LOCJ)f-CM 
CO^MCMCSI 


t^ioooooo 
NcO'-i t^co 


tOOOl^O) 
^c^c^co 


lO00'*t>LO 
CM CO -H (M fj 


Olo—iloco 

CM'*'* CM CM 


coototo 

■*rHCO.-l 


1 


>fV '■9 


MMNCOtJ' 


M'-cOlOOt^ 
rfTtcOCOM 


■*t>LnrqtD 


l>LO00tOt^ 
CO'^CONCM 


00 to 00 too 

rHCO-HCO!N 


lO00-*00'* 
CO CT* -H CO 


tOCOf-t- 
WCOCO-H 


N^iv on 




ot>tnoCTi 

tfCMTj-OCO 


COO'* eg-* 


t^COtOC^'* 
CO 00 CO-* CO 


^totooto 

'*t^rH0O^ 


f-t-ooco-* 
•*-**-*'* 


tooooo 
coc^]'*^ 


CO 


»d oj 


OOOO'HOO 

CO -g- CO (C -q- 


OOCJlOOO 

coLOTpLnco 


LOooO'^Ln 

MLOTf IN 


—1'* 05 00 05 

pqcocoioco 


Cg^t>rtCO 
-*t>COC0N 


005COtOCM 

■*-*'* CO* 


LO to 00 to 
CO CM CO CM 


Q 

»; 
o 
o 
w 


»g }d 


OCT! 00 pq CO 


'J'COlOt^CO 


OHD^tOOO 
M-*CO(N-* 


C^tOtDOO 

Tt^'^lO'* 


0)M05C0t- 
rHOOMCOCO 


t-O'^LOtO 
to to LOCO CO 


LO(N'*tD 
■* CO LOCO 


>M MD 


CO CO LOOM 

Tj-cMcoLOun 


lO-*-H00-* 

lO-^Tf^DCO 


Lnt^ot>io 

CO'* CO CO CO 


OOOCOt>LO 
C0t>-*IMCO 


CMOMOO 

NtO'**Ln 


lOO)CMNL0 
•*lOlO-*CO 


C0lOO5'H 
CO*'* CM 




}Vi »[ 


O (C Ol 'sD Tf 


incT>^Lnt> 

Tj'^LO'3'CO 


t^LOLnoico 
core CO COIN 


05 00'* 00 to 

rt-^TCc^co 


ot^rtoor- 

CMtO^CO-* 


tot-t-ioco 
TriO'*'*co 


CMOtOO 
■*LO'*CM 


o 
o 


1.0 oj 


00t>P3CT>O 

Tj- Tj- -J. !£) in 


■^OOCOrfO 


Oioorjooo 

■*tO'*C0'* 


t-.t^^lOCv) 
■*LO'*'*'* 


to 03 to t^ t- 
lOLO-*C0tO 


OOrtOOtOlO 
Lot- to to-* 


t-toooio 
-*to*co 


^a of 


lOcoLniOLfi 


t>-*ooai«o 


coco-* too 

CO t^-* COLO 


C0Ot>00C0 
IOlOtI"*'* 


to CO t^ CM to 

LO 00'* CO to 


000 0501-- 

t>to-*to^ 


OOt-O) 
■*C0'*^ 


03 01 


in Tj. (£) to U3 


I>-CqOC0C^3 


O>CT)00r^LO 

COIOCO-*'* 


OPJIOLOOO 
tolr^-^iNCO 


CMOC0t'~00 
tO00**lO 


LOCOOOt-CM 
to to to-* CO 


■*tooto 

CO to to CM 




^a ID 


.-irjtoooin 
Tj< Ti- ■* tD ;d 


CO^OOCOCTl 

■*Ln untried 


in ',o CO o lO 
^to-^coto 


MOOCOf- 
t>tO'*'*C0 


'*lO'*'*LO 

tooo*coio 


CM CM CM 00 to 
LOtOtOCT* 


'* CO to CO 




OD M3 


coLncj)t>.^ 


OOtOCTlOlCO 
M'tO'3't^-* 


'^intDrg'* 

lOt>^'*'* 


totooooto 

C^t^-^tOrP 


tO'S'lOCMCM 
to 00-*'* to 


CML0C0O5tO 
to to t- lO to 


t-ot-t- 

tO to to CO 




^J HD 


tOOOCOOOl 
tOLn<£>tr~(D 


■^CMtflt^CO 
LOIDLOOCO 


00'>*t^CO'* 

coc^-^-^c^a 


O-lOLOtOt 


COIOOOOO 

i>t>tointo 


OtOr-^OOLO 

tOtOC-LOtO 


OOtOLO 
t-t-lO'* 




»a m 


tOLOtOtXD 


OOOOOt^ 
<0 [> t£i tr^ rji 


OCOLOOCO 
■^•0^*00 


COlOlOtO^ 
tXOLOtO ■* 


t'^ to 00 05 00 

to 00-* CO c^ 


tO'*Ot>0 
to to to tot— 


OLOt-O 

to to to '* 




ns ?fV 


Ol C-- O 'X> o 


COOOOO-H 
f~t^t»00t^ 


oi>o-<to 
t^t^LO-^c;! 


^OOOCOf- 
t>05LOtO* 


00O5t-l00W 
00C^tOLOf~ 


LOOI>t-00 

t-oot-c-to 


t-LOt-lO 

t- t-t- to 




■WS "3 


LOt>-mLnco 

lO(£lt>t^tD 


loootooiin 


OOOOOILO 
y5t>tDLOI> 


lOCOLOLOlO 
00 001>t>t> 


O^L01>rt 
00 05 LO to 00 


t-005t-tO 

t-t-totot- 


00<J50 
O)05tOO) 




MH «7 


ooo t~t^-* 

CD00t>t><£) 


O O) 00 C^J 00 


LOcorjoo-* 

(OOOCOIOOO 


C0 05t>l>.05 
00t>0t01> 


t0 0001>05 

t- 00 tor- 00 


CM CO CM to CM 

00 00 00tDt35 


00 LO to CO 
OOOOt-O) 




of pv 


t> 00 00 1.^ «D 


00OOCJ5 10 


'*CT>L0t^O 

t>t><oiot> 


■^ooor-oo 

00O001>t> 


'*0050CM 
05 05O00 00 


C^LOCDOt- 
OOt-OOt-t- 


CO CO 00 't 
05 O) t— 05 




IS »n 


T)<O00 00I> 


NOLOOLO 

oooooocjit^ 


I>0001>0> 


tOC0N'*l> 
00 05 00 to 1> 


t-'*MCOCM 
C0O5CO00O5 


OOCMCOO 
00 0)0)00 05 


to CO (35 to 
00 00 t- 00 




..s*//jjgSSSf^ 


oOLnoocooi 


0-5t"LOOCO 
00 05 Ul tool 


OLOOOtOO 

oo05C^Lnoo 


tOCMLOOCO 
00 0)0) £35 00 


CMCMt-C0 05 

t-t- 05 05 00 


COLO 00 to 
00O)tOO5 



12 



TEACHERS COLLEGE BULLETIN 



In connection with this rating of compositions in class, certain 
compositions had been read aloud to the class and a general 
discussion had been held, dealing not only with the merit of 

Directions 

Do all of this work independently. Do not let your judgment be influenced 
by anyone or by anything other than the quality of the compositions themselves. 
First: 

Arrange the thirty compositions in the order of their merit, giving the best 
composition rank number i, the next best rank number 2, and so on down to the 
poorest, which should receive rank number 30. Do not give any two composi- 
tions the same rank. 

Record your judgment in the proper place on the blank below by writing neatly 
(or printing, if your writing is quality 9 or less) the identification letters of the 
composition opposite the rank number to which you have assigned it. 

Second: 

Determine as accurately as possible the position of each of the thirty compo- 
sitions on the Hillegas Scale or on the Thorndike Extension of the Hillegas Scale. 

You may assign values intermediate to those given on the scale if you feel rea- 
sonably sure of your judgment. (You may, for example, wish to assign a value 
of 05 to one composition, 10 to another, 15 to another, etc., even though there 
are no samples on the scale between o and 18.) 

Record below the value you assign to each composition, and return your record 
and material to Mr. Trabue before consulting with anyone regarding your judg- 
ments. 



Rank 
Number 


Composition 
Lettered 


Quality on 
Hillegas Sc. 


Rank 
Number 


Composition 
Lettered 


Quality on 
Hillegas Sc. 


I . . . . 


Re St. . 


82 


16 


Ri Be 


^8 


a. . . . 


. ...AdJa.... 


80 


17 


... Ca Pe 


. ?? 


I. . . . 


. . . .He Si . ■ ■ 


7n 


18. .. . 






4. . . . 


.... Eu Sw . . . 


78 


10. . . . 


Ro McN. 


2C 


?. . . . 


... .LuRh... 


7c 


20 


Ha Da 


2d 


6 


....MiSu.... 


60 


21 . . 


... Qu Fr 


2 ? 


7. . . . 


. .. .Jo Dr.... 


c ? 


22. . . . 


.... Ma Ro . 


21 


8... . 


. .. .ChCo.... 


CO 


2^. . . . 


Ma 


20 


0. . . . 


.... Lo Co .... 


40 


2A 


Jo Si 


. 10 


10. . . . 


....ChFr.... 


48 


2K. . . . 


Ro R . 


18 


II. . . . 


... .ThBe.... 


47 


26 


. . . .Pa Mu... 


17 


12. . . . 


CoO'T... 


46 


27 


. . .Le Bo 


. . 16 


i^. . . . 


....ChDr.... 


4c 


28 


Le O'B . 


Id 


14. . . . 


. ...JeMi.... 


44 


20. . . . 


....FrKo.... 


17 


IK. . . . 


Ch Wi. . . . 


42 


^0. . . . 


. . .Ru Do . . 


















SUPPLEMENTING THE HILLEGAS SCALE 



13 



the particular compositions read but also with what constitutes 
general merit in English compositions. This group of 65 judges 
will therefore be referred to during the remainder of this dis- 
cussion as the " trained group." The ratings given by these 
judges on each of the thirty samples of composition will be 
found in Record Sheet No. r. 

The second group of 74 judges were assigned the task of rat- 
ing the thirty mimeographed compositions before any discus- 
sion had been held in class either of the derivation or the use 
of the Hillegas Scale. This group of judges were asked to indi- 
cate on their record sheets whether or not they had ever used 
the Hillegas Scale before, but some of them neglected to make 
any statement regarding the matter. Where no statement 
was made, the writer grouped the records of those who had 
been in attendance at Teachers College previous to this year 
with the group who stated that they were acquainted with the 
scale, while records made by new students were grouped with 
those who reported that they were unfamihar with the scale. 
The judgments of the group who were " unfamiliar " with the 
scale are shown in Record Sheet No. 2, while the judgments 
of the " familiar " group are shown in Record Sheet No. 3. 

The median 1 rating of each of the thirty compositions is 
probably the most accurate measure of its real position on the 
Hillegas Scale. 

A graphic representation of the median value of each compo- 
sition is shown in Fig. i. An ideal scale would, of course, have 
one sample at o, another sample at i.o unit above o, another 
sample at 2.0 units above o, and so on. Not enough of the 
samples used in this study have values at exactly the right points 
to make possible such an ideal scale, but as close an approxima- 
tion to the ideal as was possible has been made. The values 
of the resulting supplementary scale are shown in Fig. i, to- 
gether with the values of the original Hillegas Scale and the 
values of all the compositions from which the samples of the 
supplement were selected. 



'The median rating is the rating above and below which there were an equal 
number of ratings, — in this case, 69 judges rated higher and 69 rated lower than 
the median rating on each composition. 



14 TEACHERS COLLEGE BULLETIN 

The reader will observe in Fig. i that two of the thirty samples 
are very near quality i, two are very near quality 5, and two 
are very near quality 8. In so far as the median ratings are 
concerned, it would apparently matter very little which of the 
two compositions in each of these three pairs was retained as a 
part of the supplementary scale. In order to determine which 
one should be used in each case, the distribution of ratings on 
each composition of the three pairs was more carefully studied. 
It was found that although both compositions He Si and Re St 
have a median rating of 8.0, the judges had greatly overesti- 
mated or greatly underestimated the value of Re St in fewer 



HILLEGAS p 

30 
Samples 



10 




Fig. I. Values on the Hillegas Scale of thirty compositions from which nine 
were selected for the Nassau County Supplement 



cases than they had the value of He Si. In other words, the 
judges had agreed as to the value of composition Re St much 
more closely than they had agreed as to the value of He Si. 
In a similar manner, although both compositions Ch Dr and 
Lo Co have a median rating of approximately 5.0, the judges 
agreed more closely as to the value of Lo Co. Compositions 
Ma and Jo Si, likewise, are both very near quality i.o, but 
there were somewhat fewer extremely bad judgments on compo- 
sition Ma. It seems quite evident that to be of the greatest 
usefulness a scale should be composed of compositions about 
whose quahty there is the least possible disagreement. In the 
three cases, therefore, where two compositions fall at approxi- 
mately the proper point on the quality scale, that sample was 
used which showed the smallest number of very bad judgments. 



SUPPLEMENTING THE HILLEGAS SCALE 



IS 



In order to see more clearly the significance of the above 
paragraph and to understand the next few pages of this dis- 
cussion, the reader should recall how the values of the Hillegas 
Scale were established. One unit on the Hillegas Scale is just 
such a difference between composition x and composition y 
as would cause 75 per cent of Dr. Hillegas' 202 original judges 
to declare that x was better than y, while the remaining 25 
per cent of the judges declared that y was better than x. If 
50 per cent had said that x were better than y and the other 
50 per cent that y were better than x, then we should have 
been safe in saying that x and y were equal. A difference 
noticed by 60 per cent but not noticed by 40 per cent would 
probably be a real difference, but it would not be a very useful 
difference, for 4 persons out of 10 would not be able to recognize 
and employ the difference. A difference which is noticed by 
3 out of 4 persons is, however, a really useful difference, and it 
also agrees with some of our statistical concepts. 

Dr. Hillegas assumed, and there is every reason to believe 
that the assumption is perfectly sound, that judgments as to 
the quality of a composition will be distributed symmetrically 
on either side of its true value, — that just as many will over- 
estimate as under-estimate its value, and that large errors in 
either direction will balance each other and be much less fre- 
quent than small errors. If the true value of composition x 
is represented in Fig. 2 by the point marked o on the base line, 
the judgments as to its value would, according to the assump- 
tion, be distributed both to the right and to the left of o in 
diminishing frequencies represented by the surface enclosed 
between the heavy curved line and the base. This symmetrical 
surface representing the judgments on composition x, whose 
true value on the base line is o, is the well-known friend and 
servant of the statistician, known as the Normal Surface of 
Frequency. 

Another assumption made by Dr. Hillegas is that the variabil- 
ity of judgments on one composition is exactly equal to the 
variability of judgments on another composition. In Fig. 2 
let us suppose that a second composition y has a true value on 
the base line which is just sufficiently far below (to the left of) 
the true value of composition x that exactly 75 per cent of the 



i6 



TEACHERS COLLEGE BULLETIN 



judges in distribution x assigned to x values above (to the right 
of) the true location of y. Then, if judges vary just as widely 
on one composition as on another, we should expect to find 75 
per cent of the judges in distribution surface y rating y as below 
the true value of composition x on the base line. The sta- 
tistical name for the difference between the true value or median 
(50 per cent) point of a distribution and the point on the base 
which has 25 per cent of the distribution on one side and 75 
per cent on the other is the Median Deviation (M.D. or P.E.). 
The second assumption may therefore be restated to read, 
" The Median Deviation of a distribution of judgments on one 
composition is equal to the M.D. of a distribution of judgments 
on another composition." 




-5-4-3-2-10123456 
Fig. 2. Assumed equal variability of judgments on all compositions 

The statement may now be made, which might have been 
made at the very beginning of the discussion for those who are 
familiar with statistical terms, that the Median Deviation is 
the unit of difference employed in the Hillegas Scale. The 
sample composition which on the scale has a value of 2.6 was 
judged as better than 2.6 by 50 per cent and as poorer than 2.6 
by the other 50 per cent of Hillegas' 202 judges. Twenty-five 
per cent of these judges would have judged it as poorer than a 
sample whose actual value was 1.6, and on the other extreme 
25 per cent of these judges would have judged it better than a 
sample whose real value was 3.6. We may assume that if 
Dr. Hillegas' judges had included in their Hst the composition 
Re St used in this study, 50 per cent of them would have rated 
it as better than 8.0 and the other 50 per cent would have rated 
it as poorer than 8.0, for the median of 139 judgments places 
this sample Re St at 8.0. If the M.D. of judgments on one 



SUPPLEMENTING THE HILLEGAS SCALE 



17 



composition is equal to the M.D. of judgments on another, we 
may further assume that 25 per cent of Dr. Hi]Ie;:^as' Judges 
would have rated Re St as poorer than 7.0 and another 25 per 
cent of them would have rated it as better than 9.0. It is 
quite surprismg, therefore, to find that only about 10 per cent 
of the 139 judges used in this study rated Re St as low as 7.0 
and only about 10 per cent rated it as high as 9.0. 

If the base line of the two curves in Fig. 3 is allowed to repre- 
sent M.D. units on the Hillegas Scale, the low rather fiat curve 
encloses the surface representing the general distribution of 
judgments to be expected (according to Dr. Hillegas) for any 



Re St 



Gsneral dls- 
trfbut ion & 
scale 




r Re St ao&le 
— HeSl ecale 



Fig. 3. Small variability of judgments on one composition 



sample of composition, while the narrow high curve encloses 
the surface representing the actual distribution of judgments 
on composition Re St. whose median value was found to be 8.0. 
If the M.D. of the Re St distribution were used as the unit of a 
scale, in place of the M.D. of an assumed distribution common 
to all compositions, note how much shorter these scale units 
would be and how many more units would therefore be included 
between any two different points on the scale. Fig. 3 also shows 
the M.D. units that would result from using as the basis for a 
scale the distribution of judgments on composition He Si, whose 



i8 



TEACHERS COLLEGE BULLETIN 



median value was also found to be approximately 8.0. If the dis- 
tribution on He Si had been shown it would have been only slightly 
higher and less wide than the general distribution assumed by 
Hillegas. Such large differences in the form of surfaces of 
distribution of judgments on two equally good compositions 
were certainly unexpected. They tend to make one critical of 
the assumption that the variability of judgments on one com- 
position is equal to the variability of judgments on any other. 
A rough distribution is given below in Table I of the 139 
actual deviations from the true value for each composition in 
the three pairs of equal samples mentioned above. 

TABLE I 



Composi- 
tion 
called 


Median 
value 


DiSTRIB. 

FROM 


OF Amounts of Deviation 
Final Median Value 


Total 




-S 
to to 

•4 -9 


i.o i.s 2.0 2.5 3.0 3.S 
to to to to to to 
1.4 1.9 2.4 2.9 3.4 3.9 


devia- 
tion 


He Si 
Re St 


8.01 
8.00 


47 30 
62 30 


31 IS 9 4 2 I 
37 S 2 I I I 


139 
139 


.82s 
.487 


Ma 
Jo Si 


1.06 
1.03 


S3 67 
SI 60 


18 I 

22 s I 


139 
139 


.627 
.S68 


ChDr 
Lo Co 


4.98 
4-97 


28 43 
42 46 


30 23 12 21 

27 IS 6 2 I 


139 
139 


.900 
.766 



' The median of these deviations was calculated from a much finer grouping than that 
shown in this table, hence the unusual M. D on Ma. 



In connection with this table it was observed that the M.D. 
of judgments for each of the six compositions was less than one 
unit (i M.D. of Hillegas' judges). This fact had in it a sugges- 
tion that possibly the 139 judges used in this experiment agreed 
with each other more closely than the judges used by Dr. Hille- 
gas agreed among themselves. To test this possibility, the 
method used by Hillegas on the judgments of 202 judges was 
employed upon the judgments of the 139 judges used in this 
study. The resulting calculation is shown in Table II. 

Table II, beginning at the top and reading across horizon- 
tally to the right {rf. Original Record Sheets 1,2, and 3) reads 
as follows: Comparing the values given by each individual 



SUPPLEMENTING THE HILLEGAS SCALE 



19 



TABLE II 

Result of Applying Hillegas' Method to 139 Ratings on the 
HiLLEGAS Scale 





No. of Records 




Amount 






showing 


Percent- 


"Poorer" 


Amount better 


Relation of 




age 


( ^ ) 


than Ru Do 






"Is "Is 


"Better" 


\ M.D. / 


(i=M.D.) 




Poorer" Better" 








Ru Do to Ma 


^33 6 


4-3 


2-55 


Ma is 2.5s 


Ma to Le Bo 


131 8 


5-8 


2.34 


Le Bo is 4 . 89 


Le Bo to Ma Ro 


120 19 


13-7 


1.62 


Ma Ro is 6.51 


Ma Ro to Ca Pe 


117 22 


15-8 


1.48 


Ca Pe is 7.99 


Ca Pe to La Co 


"4 25 


18.0 


1.36 


Lo Co is 9.3s 


Lo Co to Th Be 


119 20 


14.4 


1.58 


Th Be is 10.93 


ThBe to Mi Su 


129 10 


7.2 


2.17 


Mi Su is 13 . 10 


MiSu to Re St 


123 16 


"•5 


1.78 


Re St is 14.88 



judge to compositions Ru Do and Ma, 133 gave Ru Do a 
lower rating than Ma; while 6 of the 139, or 4.3 per cent, 
gave Ru Do a higher rating than Ma; converting 4.3 per 
cent into terms of M.D. by means of the table used by Pro- 
fessor Hillegas, there are 2.55 M.D. units between Ru Do 
and Ma. Similarly, 8 of the 139 judges, or 5.8 per cent, decide 
that Ma is better than Le Bo, so it is therefore 2.34 M.D. worse 
than Le Bo. Le Bo being 2.34 units better than Ma, which is 
2.55 units better than Ru Do, we must conclude that Le Bo is 
4.89 units (2.55 + 2.34=4.89) better than Ru Do. It will be 
observed that Re St, the best composition, is 14.88 units above 
Ru Do. It will be remembered that the median values of these 
two compositions on the Hillegas Scale differ by exactly 8.00 
units. The difference in quality between Ru Do and Re St 
actually remains the same, but if one accepts the M.D. of the 
139 judges as a basis he finds 6.88 units more between them 
than he finds if he accepts the M.D. of Hillegas' 202 judges. 
This difference is shown graphically in Fig. 4. 



RD M LB MR CP 
• • • • • 

I I I I I 



LC TB 



MS RS 



I I I 



Median RATINGS vs. 

Calculated VALUES 



RD 



LB 



MR 



CP 



LC 



TB 



MS 



RS 



I I I I I I 



O 1 
Fig. 4, 



2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 

Linear projections of median ratings and of M.D. values for samples 
included in Nassau County Supplement 



20 TEACHERS COLLEGE BULLETIN 

The discrepancy between the two groups of judges may be 
due to the fact that the 139 judges had the opportunity of using 
an objective scale (the Hillegas Scale) in their arrangements 
of the thirty compositions, while the origina 202 judges had no 
common objective measure of merit in composition. The 139 
judges may agree more closely with each other because they 
were all " school people." The fact that the thirty composi- 
tions were all of the same general type may have been partially 
responsible for the greater amount of agreement as to differ- 
ences in merit. Probably all of these factors combine with 
less obvious causes to produce greater agreement among the 
139 judges in this experiment than existed among Hillegas' 
202 judges. 

In order to see whether the four hours' discussion and use of 
the Hillegas Scale in the summer session class had resulted in 
any increased ability to agree as to composition quality, Dr. 
Hillegas' method was used on the 65 " trained " judges' judg- 
ments and on the 74 " untrained " judges' judgments separately. 
The results of these calculations are shown in Table III. The 
reader will recall again that Re St is only 8.00 units above 

TABLE III 



Measured by 



A. Median on Hille- 
gas Scale 

B. M.D. of 74 Un- 
trained Judges 

C. M.D. of 65 Trained 
Judges 



Distance above zero of samples in Nassau County 
Supplement 



















^ 




C) 






CQ 


(< 


0. 


c"> 


cq 


^ 


c^ 


0=; 









c3 




-1 


S 


% 





o 1.06 1.92 2.80 3.84 4.96 6.00 7.21 8.00 

O 2.22 4.17 5.41 6.87 8.33 9.71 11.66 13.39 

o 3.2s 6.50 8.79 10.30 II. 55 13.38 15.88 17.71 



Ru Do, if we use the M.D. of Hillegas' judges as the unit, while 
it is 13.4 units above Ru Do if the " untrained " group are used 
as the basis, and if the 65 " trained " judges are used as a basis 
the difference is increased to 17.7 units. As was said before, 
the " trained " group were probably not more capable by nature 
than the " untrained." Four hours' discussion and use of the 
scale had merely made them agree much more closely in their 



SUPPLEMENTING THE HILLEGAS SCALE 2 1 

judgments of composition quality, even though they rated the 
thirty compositions without ever having seen any of them be- 
fore and without conferring at all about them. These differ- 
ences are shown graphically in Fig. 5 for each interval between 
samples of the supplement. 

RD-M M-LB LB-MR MR-CP CP-LC LC-TB TB-MS MS-RS 

1 ^^ 1.0c Hi M ^m -88 ■^1.04 ^■■1.12 ^HililA ■^■1.21 m -79 



Fig. 5. Influence of training upon discrimination of differences between samples. 

In concluding this discussion of the derivation of the Nassau 
County Supplement to the Hillegas Scale, the writer must 
explain that he does not advocate discarding the Hillegas Scale. 
It was impossible, before the Hillegas Scale was devised, to know 
that judges would be more variable on one composition than 
on another. Now that we can measure such differences in 
variability fairly well, we should get together a large number 
of compositions upon which judges do agree very closely. Just 
because the median judgment places a sample at the right point 
on the quality scale is not sufficient excuse for using it. Only 
those samples upon which people agree closely should be used. 
In order to get a very perfect scale, it would possibly be well 
to train the judges until additional training would not result 
in greater agreement among them. If the variability of human 
judgment is not a constant on which our scales for comparison 
may be built, possibly the variability of the judgments of col- 
lege graduates having ten hours' training in doing this work 
would more nearly approximate a constant for reference. For 
all practical purposes at present, however, some form of the 
Hillegas Scale, crude though it is, will supply all our needs in 
this direction. 

The need for the second supplement, a tentative statement 
of standards to be achieved in English composition, was empha- 
sized very strongly by the differences between the results being 
obtained by the same grades in different schools. To find fourth 
and fifth grade classes acliieving results distinctly superior to 
eighth grade classes in a neighboring school is rather surprising 



22 



TEACHERS COLLEGE BULLETIN 



until one stops to consider that up to the present no one has 
told these teachers what quality of English composition to 
accept in their classes. Teachers probably determine their 
standards most commonly from the compositions pupils hand 
in during the first few meetings of the class. In very rare in- 
stances samples are preserved from year to year and used as a 
basis for grading compositions from later classes. In the large 
majority of cases, however, a new teacher has no way of learn- 
ing what to expect or require. The random results which she 
obtains are illustrated by the records of the eight classes in- 
cluded in Table IV. Two of these classes are fourth grades, 
two are fifth, two are eighth grades, two are senior classes in 
high school, and all of them are in the same county within ten 

miles of each other. 

TABLE IV 

Records of Eight Classes 





No. of compositions rated on sample of 




Median 


75% 


In 


value 


Total 






class 




No 






o 1.8 2.6 3.7 4.7 5.8 6.7 7-7 




Quality 


A 


3 27 9 


39 


3.80 


3-41 


B 


I 14 8 I 


24 


2.94 


2 


55 


C 


6 8 12 4 


30 


6.37 


5 


48 


D 


I 8 19 4 


32 


5-66 


5 


16 


E 


S 23 19 I 


48 


1.99 


I 


31 


F 


34 7 I 


42 


3-8i 


3 


48 


G 


6 6 I 


13 


6.38 


S 


84 


H 


12 17 5 


34 


3-46 


2.87 



A and H are fourth grades, E and F are fifth grades, B and C are eighth grades, and 
D and G are fourth year high school classes. 

The above table reads from left to right as follows: Three 
pupils in Class A wrote compositions which were scored on 2.6, 
twenty-seven pupils wrote compositions which were scored on 
3.7, and nine pupils wrote compositions of quahty 4.7, making 
altogether thirty-nine compositions, half of which were of 
quality 3.8, or better, and 75 per cent of which were of quality 
3.41 or better. 

Notice in the above table the difference between the two 
eighth grade classes or between the two fifth grade classes. No- 
tice that a fourth grade class (A) does work distinctly better 



SUPPLEMENTING THE HILLEGAS SCALE 



23 



than an eighth grade class (B), and that a senior class in high 
school (D) is quite clearly inferior to an eighth grade class (C). 
Certainly no teacher would be satisfied to have her class doing 
work typical of lower grades unless she were ignorant that they 
were doing such work. Knowledge of how well a class should 
do and ability to know when it was actually doing that well 
would seem to be prerequisites for successfully teaching a 
subject. 

The purpose of the standards proposed in the following para- 
graphs is merely to serve as a guide to teachers, so that they 
may not be left as they have been in the past without any 
definite idea of the achievements to which their classes should 
aspire. Better standards will doubtless be proposed by others, 
but the suggestions which follow will be worth while if they 
serve merely to call forth criticism and more satisfactory 
standards. 

How well do " Seniors " in high school write? What quality 
of compositions would you accept from " Sophomores " in high 
school? These two questions were asked of advanced students 
in Educational Administration. The answers were expressed 
in each case by means of the list of thirty compositions employed 
in the first part of this study. Without knowing the exact value 
of any of the thirty compositions, thirty-six experienced super- 
visors and administrators each selected two compositions from 
the lot, one to represent the average or median achievement 
of high school senior classes, and the other to represent just 
barely passing quaHty for the same grade. Forty-six other 
members of the group selected two compositions each, one to 
represent the average quality obtained from sophomore classes 
in high schools they had known, and the other to represent 
just barely passing or 70 per cent compositions for the same high 
school classes. The distribution of estimates made is shown 
in Table V. 

It would not be sufiicient, however, to take the estimates 
of even the most able group of educators in the world as a 
basis for standards of achievement. It is necessary to know 
in addition just how well pupils actually write. The median 
results of composition tests and measurements on the Hillegas 
Scale are shown in Table VI for a number of typical school 



24 TEACHERS COLLEGE BULLETIN 



ESTIMATI 


iS OF Quality 

Sample 
He Si 


TABLE V 

OF English Compositk 
IN High Schools 

Senior classes 


3NS TO BE Exp: 

Sophomore 

Median 

(8S%) 

.... 2. . . 


ECTED 

classes 


Value 
8.0I 


Median 
(8S%) 


Passing 
(70%) 


Passing 

(70%) 


8.00 


Re Si 


. . .3 




1 




7-68 


Adja 


. . .3 




c 




7.51 


LuRh 


. . .9 


2 


, 7 




7.47 


Eu Sw 


...a 


I 


... 2 . . 


2 


7.22 
6.01 


Mi Su 

ThBe 


•••9 

. . .1 


6.. . ! . 


13 


... .2 
8 


5-89 


ChFr 


. . .4 


3 


8 


II 


5.76 


ChCo 


. . .4 


8 


I . 


Q 


4.98 


ChDr 




4 




.... 2 


4.97 


LoCo 




c 


2 




4.86 


JoDr 










4-77 


CoO'T 




2 


2 




4.36 


JeMi 




2 






4.16 


ChWi 








2 


4-03 


RiBe 










3-84 


CaPe 




. .... I 






3.66 


RoMcN.... 




I 




1 


3.31 


Gr Mi ' 










2.81 


MaRo 










2.67 


QuFr 










2.54 


Ha Da 




I 






2.50 


PaMti 










2.40 


LeO'B 










1-93 


LeBo 










1.06 


Ma 










1.03 


Jo Si 










.80 


RoR 








I 


.68 


FrKo 










.004 


RuDo 











Number of judges 36 36 46 46 

Median 7.35 5.49 7.12 5.83 

systems. The lower line of this table shows the tentative 
standard medians proposed by the writer as the result of this 
study. Expressed briefly, the proposed standards show the 
quahty of compositions to be expected from at least half of a 
normal class of American boys and girls at the end of any 
given school year. 

Some of the medians given in Table VI are shown graphically 
in Fig. 6. 

It will be observed in both the table and the graph that the 



SUPPLEMENTING THE HILLEGAS SCALE 



25 



TABLE VI 

Median Achievements in English Composition by Grades 
(Hillegas Scale) 



School system 



Median score attained in grade 



IV V VI VII VIII '"^ ^^ 3rd 4th 
yr. yr. yr. yr. 



Lead, South Dakota^ 

Newark, N. J. (i school)^ 

Ethical Cult. Sch., N. Y. C.\ 

Chatham, N. J^ 

Salt Lake City, Utah* 

Butte, Montana* 

Nassau County, N. Y.' 

South River, N. P 

Mobile County, Alabama* . . . . 

Mobile, Alabama* 

Fifty -four High Schools* 

Tentative Standard Medians . 



357 411 4 
2-39 2.51 3 
4.01 4 
2-95 2.85 4 
3.58 3.84 4 
2.34 2.80 3 
2.76 3.42 3 
2-31 2.55 3 
3.20 3.91 4 
331 3-^5 4 



■H 5 


or 5- 


5^ 4 


33 5- 


72 5 


39 5- 


10 4 


02 5. 


.61 5 


16 6. 


■41 3 


77 4- 


.82 4 


18 4. 


.78 4 


75 5- 


■34 4 


22 


.60 4 


95 



57 

27 

74 High 

29 School 

37 

II 

56 s.oo 5.2s 5.68 5.94 

62 5. 18 5.02 5.95 6.30 
5.56 6.38 6.05 6.77 
6.69 6.93 7.24 7.54 
4.99 5.88 6.38 6.69 



3.5 4.0 4.5 s.o 5.5 6.0 6.5 6.9 7.2 



' From Annual Report of Superintendent of Schools, 1915-16, Lead, S. D. 

'' From unpublished studies made in the department of educational administration. 
Teachers College. 

3 Calculated from the distributions published in Cubberley's School Organization and 
Administration. World Book Company, Yonkers-on-Hudson, N. Y., 1916. 

•• From Strayer's Some Problems in City School Administration, World Book Com- 
pany, Yonkers-on-Hudson, N. Y., 1916. 

6 From unpublished report of the Mobile School Survey by the Bureau of Municipal 
Research of New York City. The Mobile County scores are for the white pupils outside 
the city, while the Mobile scores are for the white pupils in the city schools of Mobile. The 
seventh is the highest elementary school grade in the Alabama schools. 

"From master's essay of Albion Hale I3rainard, prepared m the department of educa- 
tional administration, Teachers College, 1916. High schools from over 35 different states 
are included. 

standard is higher than the majority of the schools have actually 
achieved, although at each grade one or more school systems 
have shown higher achievements than the standard suggested. 
If some schools are able to meet this standard without making 
particular efforts to do so, many schools should be able to meet 
it when they make definite efforts to do so. It seems desirable, 
at least, to set high standards " to shoot at " rather than to 
make the standards conform more closely to the achievements 
which the present more or less haphazard teaching efforts 
secure. 

There is, in addition to the above, a real value in having a 
secondary standard to indicate the width of the distribution of 
scores to be expected in a class. There will usually be a small 



26 



TEACHERS COLLEGE BULLETIN 



8.00 



7.00 



6.00 



5.00 



4.00 



S.OO 



2.00 



4 



SCHOOL GRADE 
7 8 9 10 11 12 

















^^ 










/ 


/ / 
/ / 


X 


2^ 






A 


/ ■'/y 




% 


/ 


"" 


/ 


/ 


6, 


> / 


s*/ 










/ 































NC-N»SiAUCa 



Fig. 6 



number of pupils in each class who will be unable to approach 
the median standard, and for them a lower " mark " may act 
as a stimulus. For this purpose, the writer suggests that 75 
per cent of each class ought to write compositions of or better 
than the quality indicated below: 

Quality of English Composition to be Equalled or Excelled by 
AT Least 75 Per Cent of the Pupils in a Class 

In grade IV V VI VII VIII IX X XI XII 



Quality. 



30 3-5 



VI 

40 4.5 



S.o 5.5 6.0 6.4 6.7 



SUPPLEMENTING THE HILLEGAS SCALE 



27 



The qualities suggested above for 75 per cent of a class are 
in each case just .5 of a unit lower than the qualities suggested 
for 50 per cent of the class. This difference between the achieve- 
ment of half and the achievement of three-fourths of a class is 
practically the difference one actually finds in classes as they 
are at present. The chief sources of proof for this statement 
are the tables of distributions in Nassau County, New York, 
where the score on each paper was practically the median of 
independent ratings by three well-trained judges. It would 
be somewhat unsafe to use distributions based on a single rating 
of each paper by an untrained person. Taking the combined 
scores from thirty-three Nassau County schools and consider- 
ing the distributions of results as though each grade were a 
single large class, differences were calculated by the method 
indicated in Table VII. 

TABLE VII 

Differences Between Achievements of 50 Per Cent and of 75 
Per Cent of a Grade in Nassau County 



School grade 


IV 


V 


VI 


VII 


VIII 


IX 


X 


XI 


XII 


No. of compositions . 
Median (50%) score. 
75% of grade above. 


1046 
2.76 
2.25 


930 

3-42 
2.72 


894 
3-82 
3.16 


741 
4.18 

3-49 


607 
4-56 
3-79 


554 
5.00 
4.42 


332 
5-25 
4.60 


237 
5.68 

4-95 


i6s 
5-94 
5-42 


Difference (50-75).. . 


•51 


.70 


.66 


.69 


• 77 


•58 


•6S 


•73 


•52 



As was shown earlier in this discussion, being in the same 
grade in different Nassau County schools does not mean doing 
the same sort of English composition work. The above differ- 
ences are larger than would be found if we were dealing with 
actual classes or recitation sections. By taking all those recita- 
tion sections in Nassau County containing more than 25 pupils 
in the fourth grade, 20 pupils in the eighth grade, and 12 pupils 
in the senior high school grade, the distributions of differences 
between the achievements of 75 per cent and the achievements 
of 50 per cent of the pupils in actual classes were found to be 
as shown in Table VIII. 

From Table VIII it will be observed that the requirement 
is not too high, which suggests that the achievement of 75 per 



28 



TEACHERS COLLEGE BULLETIN 



TABLE VIII 

Frequency of Differences Between Achievements of 50 Per Cent 
AND OF 75 Per Cent of Actual Classes 



Size of Differences 


In grade In grade In grade 
IV VIII XII 


. ^0—. ■?Q Hilleeas units 


3 3 

5 5 2 

4 3 2 
7 I I 

2 2 

3 I I 


.40-. 49 


. KO-. ^Q 


. 60-. 60 


•70-. 79 

.80+ 




Median difference' 


■59 -46 .52 





' Calculated from actual cases rather than from above distributions. 

cent of a class should be within .5 of a unit of the achievement 
of 50 per cent of the class. 

For the aid of those who may be unfamiliar with the method 
of calculating the 50 per cent or the 75 per cent achievement of 
a class, when the scores have been assigned to the papers merely 
in terms of the nearest samples on the scale, the following table 
is given. 

TABLE DC 

To Facilitate the Calculation of the Achievements of English Composi- 
tion Classes Measured by the Nassau County Supplement to the 
HiLLEGAS Scale 



Sample Actual value Range of step' Length of step Sample 



I.I 






. . . . . . 
. . 1 . 06 . . . . 


.... 0- .53 ... 
54-1 -49- ■• 




-53 
.96 




.. . . 
. ... I.I 


1-9 

2.8 






..1.93.... 
..2.81.... 


. . . .1.5CJ-2.36. .. 
•- --2. 37-3. 32... 




.87 
.96 




.. ..1.9 
....2.8 


3-8 
S-O 






..3.84.... 
..4.97.... 


....3.33-4.40... 
.. ..4.41-5.48. .. 




1.08 
1.08 




....3.8 
....S-o 


6.0 

7.2 






. .6.01. . . . 
. .7.22. .. . 


. . . .5.40-6.61. . . 
. . . .6.62-7.61. . . 




I -13 
1. 00 




....6.0 
....7.2 


8.0 
9-0 






. .8.00 

..9.0 .... 


7.62-8.50. .. 

....8.51- ... 




.89 




....8.0 
....9.0 



' The third decimal point is omitted from these tables. 

To show the method of using this table we may take the 
following distribution of scores as an example, and calculate 
the achievement of 50 per cent and of 75 per cent of the class. 

Scored on o i.i 1.9 2.8 3.8 5.0 6.0 7.2 Total 

Frequency 3 16 25 4 48 



SUPPLEMENTING THE HILLEGAS SCALE 



29 



The median achievement will be the quality above which 
there are 24 papers (48 -^ 2=24) and below which there are 24 
papers. If we count the three which were rated on 1.9 and the 
16 which were rated on 2.8 we will have only 19 papers, and in 
order to reach the middle of the distribution we shall need to 
count 5 of the 25 papers rated on 3.8. We may assume that 
the 25 papers are spread evenly over step 3.8, extending (see 
Table IX) from ^.;^t, up to 4.40 at equal intervals. If we count 
out 5 of the 25 we shall pass over one-fifth of this step which 
is 1.08 units long; in other words, we shall have gone upward 
from T,.^T, a distance of .22 of a unit (1.084- 5=. 2 2) and we shall 
therefore have arrived at quality 3.55, which is the median 
quality of the distribution. 

In order to find the achievement made by 75 per cent of the 
class, we shall need to count only one-fourth of the way into 
the distribution, — that is, we shall need to count out the lowest 
12 pupils (48-^-4=I2). After counting the 3 papers scored on 
1.9 it will be necessary to count out 9 (12 — 3=9) of the 16 
scored on 2.8. Assuming that the quaHties of the papers are 
distributed evenly over step 2.8 from quality 2.37 to quality 
3.32, we shall count out nine-sixteenths or .56 of the step, which 
is .96 of a unit long. This will take us to a point .54 of a unit 
(.96X.56=.54) above quality 2.37, which will be quality 2.91, 
the achievement made by 75 per cent of the class. 

The table for the original Hillegas Scale which corresponds 
in its use to Table IX is given here as Table X. 

TABLE X 

To Facilitate the Calculation of Class Achievements on the 
Hillegas Scale for English Composition 

Step No. Value of sample Range of step Length of step Step No. 



o 0.00- .91 91. 

1.83 92-2.21 I -SO- 



2 2.60 2. 22-3.14 93 2 

3 3 69 3. 15-4. 21 1.07 3 

4 4-74 4.22-5.29 1.08 4 

5 5 85 5-3^6.30 1. 00 S 

6 6.75 6.30-7.23 93 6 

7 7 72 7.24-8.05 81 7 

8.38 8.05-8.87 82 8 

8 9.37 8.88- 9 



30 



TEACHERS COLLEGE BULLETIN 



It is not proposed that any teacher should use the Hillegas 
Scale in any form as the basis for all her marks in EngUsh com- 
position. The scale is a measuring instrument and not a peda- 
gogical device. In her instruction a teacher may at one period 
be laying stress on punctuation and give her class marks accord- 
ing to their success in punctuation. At another period the 
teacher may be placing her emphasis upon the choice of words, 
and the class marks may well reflect this emphasis. But at 
some two or three periods during the year teachers should meas- 
ure the general results that are being obtained by this instruc- 
tion. For this purpose a number of teachers should cooperate 
with each other in rating papers on. some form of the Hillegas 
Scale. When such practices become more general, we may hope 
that results in English composition writing will become uni- 
formly better. 



APPENDIX 

More than five years have passed since the above discussion 
was first published. During this period a considerable amount 
of real progress has been made in the science of measuring the 
quality of English compositions. Professor Earl Hudelson^of 
the University of West Virginia has been one of the most fruit- 
ful workers in this field.^ 

He has found, for example, that pupils write compositions of 
a much lower quality when they use the " What I Should Like 
To Do Next Saturday " topic than when they write on such a 
topic as " My First Lie." The average results obtained from 
" My First Lie " are one full unit better than the average results 
from the topic on which pupils in Nassau County wrote. If 
an eighth grade teacher secures from her pupils compositions 
on the shorter topic, she should subtract i.oo from their median 
score before comparing it with the records made by pupils on 
the " Next Saturday " topic. Such facts indicate the necessity 
of having a uniform topic or set of topics for testing ability 
in English composition, and the importance of following uniform 
directions. 

Directions for Obtaining Compositions 
Have each pupil clear his desk and provide himself with pen, 
ink, a sheet of the usual composition paper and a blotter. The 
pupils should be asked to write (at the top of the sheet) their 
names, ages (at last birthdays), and school grades. If papers 
from more than one room or school are to be scored at the same 
time, it would be well to obtain from each pupil before the test 
begins a written record that will fully identify his paper. While 
the pupils are writing their names and other desired informa- 
tion, the examiner should write one of the four following topics 
on the blackboard. Be sure to write plainly and where all 
can see. 

The Funniest Thing I Ever Saw 

The Saddest Event of My Life 

The Happiest Hour of My Life 

The Most Unpleasant Hour of My Life 



'Hudelson, Earl, Aims, Methods and Measurements in English Composition. 



32 TEACHERS COLLEGE BULLETIN 

Call attention of the pupils to the topic in the following words : 
" I have written on the blackboard the topic for a story. You 
are to write a story about the [topic]. You may make up a 
story if you need to, but most of you can tell a real experience. 
You may have twenty minutes in which to write. See how 
interesting a story you can tell." 

All papers should be collected at the end of twenty minutes, 
whether the pupils have finished writing or not. However, in 
assigning scores to the papers, nothing should be subtracted from 
any pupil's mark because of an incomplete story. Each paper 
should be marked on the quality of the writing actually done. 

Compare the general quality of the pupil's composition with 
the general qualities of the various samples on the scale. Assign 
to each composition the numerical value of that sample on the 
scale which most nearly equals it in general merit. Go up the 
scale until you reach compositions that are distinctly better than 
the one being rated ; go down the scale until you find composi- 
tions that are distinctly poorer than the one being rated ; and 
then compromise on some value between these two limits. In 
order to secure accuracy, it would be wise to have each compo- 
sition rated on the scale by at least three competent judges, each 
making his rating without knowledge of what the others have 
assigned. The average or the middle rating of the three judges 
may be taken as the one to represent the final value of the 
composition. 

Directions for Improving the Judgments of Teachers 
In order to improve the abilities of teachers to judge the gen- 
eral qualities of English compositions, the other twenty-one 
compositions (rated by the 139 judges but not included in the 
Nassau County Supplement) are reprinted here as practice ma- 
terial. Let each judge read one of these compositions, compare 
it with the samples on the scale, decide carefully the numerical 
value that seems appropriate to it, and then turn to the Key on 
the last page of this booklet and compare his judgment with 
the value quoted there. 

One should not be satisfied to discover that he has over- 
estimated or under-estimated a composition, but he should care- 



SUPPLEMENTING THE HILLEGAS SCALE 33 

fully re-examine the composition and the scale to see why the 
median of 139 judgments gives it the value that appears in 
the Key. After discovering why one has made an error, he may 
try again on another composition, preferably not the one imme- 
diately following, because of the danger that he might remember 
its value from the Key. 

A careful rating of ten or twelve of these practice composi- 
tions, followed in each case by a comparison with the Key and 
a thorough re-examination of the scale to see why one's ratings 
differ from those in the Key, should reduce one's " personal 
equation " considerably. One may consider himself a reasonably 
competent judge when the average difference between his ratings 
and the true values has been reduced to less than .5 of a unit. 

The true values of the following compositions are given in 
the Key on page 38. 

Ri Be 4.03 

We have all ready secured bases and other things to have a good game of base- 
ball and if we finish the game in time we will go into the woods and make a hute 
and have a kind of party. We are going to have cake and crakers. We hope to 
have a good time for we are going to have games of all sorts , 

Ch Co 5.76 

My desire for enjoyment next Saturday would be a camping out party where 
I could obtain plenty of fishing, hunting and water sports, of which I am fond of. 

I would like to have my cousins accompany me, as they know a great deal 
about fishing, and know how to track wild game, owing to their previous expedi- 
tions which they made with their uncle last summer. 

It seems to me that our journey would be all the more pleasanter if we traveled 
by canoe, and make one or two portages when possible. 

I think this would be a pleasant way to spend next Saturday, as nothing is 
more injoying than to be out in the open country, where you can breath the fresh 
air of the woods and feel at home once more. 

Ha Da 2.54 

I intend to go next Saturday is to sleigh riding, and part of the time to go ice 
skating if there is any ice skating, do my lessons, do my other work, and help 
my mother^ help my brother, and sister. 

And do my poia lessons, and take my ponia lesson, and go down celler and 
chop wood for my mother, go the to the store for errands for my mother, and 
wipe the dishes, and bring up coals, wood for my mother, if I can make a kite 
and when it dries I will fly it for a little while and come in if it get cold and get 
warm and go out for a nother Httle and do the same thing if there is any more 
thing to do I wiU do it. 

Jo Dr 4.86 

Next Saturday I should like to Play ball in the morning till about 10 o'cloack. 
Then I should like to go to the beach and go in for a swim. In the Afternoon 
I should like to go to the city and see the ball game between the giants and Phila- 
delphia. In the evening I would like to go to the the theatre. 



^4 TEACHERS COLLEGE BULLETIN 

Ch Dr 4.98 

If the weather be clear, my intentions are all planned out for next Saturday. 

In the morning I have made a promise to pitch, (for a team just selected of 
fairly good players), who are to play the " Happy Nine " of Bullshead. The 
game is called at nine o'clock and the players are to be there about eight, so as 
to give us a chance to get warmed up (as base ball players calle it). 

I am a pitcher that can not be found every day of the year. As I have played 
in a great many games of base ball. I have been practicing all this week after 
school hours. And think I can do the trick. 

Ch Fr 5.89 

Next Saturday I should like to have a good game of baseball because I like that 
game very much. 

I would not care to win a game by unfair means nor by a large score but some- 
thing like one to two or one to nothing in our favor with every man watching his 
own game and playing to the best of his ability. 

I IDie baseball because you have to be quick to play a good game and keep 
your head when you are playing against odds or when your are in a tight place — 
for instance You are two runs ahead of your opponent and they have two men 
on base and a good batter up to bat in the last half of the nineth inning, — then 
a good many pitchers and players get discouraged thinking there are no chances 
to win, but the ones that stick to it until the last man is out is the man or team 
for me. 

Qu Fr 2.67 

I am going to play ball and help my mother and I am going riding, deliver papers 

then with my money go by a new ball. 
then I will feed my rabbits and chickins after they are feed I am going to Scouts 

an practice 
then when I come home I'll go play. 
At night go to the movies after movies go get a nickels worth candy 

Ad Ja 7.68 What I should like to do when I leave High School 

My highest ambition after I leave high school is to enter coUege and successfully 
complete my course there and then enter upon a career of service for others. 
From a child I have always longed to be among the poor and lowly not at first 
for the purpose of doing missionary work but because my life had had so much 
happiness that I can not bear to think of anyone being unhappy. 

I have always had a great admiration for missionaries but their lives seemed 
to be so set apart, so far above anything I could become or hope to become. Not 
until a few months ago did the idea come to me with any great force but from 
that time on I was determined to become a missionary not only that I may share- 
my happiness with others, but to teach them of the loving God, believing that 
through Him, they may be brought into lives of happiness and usefulness. 

Fr Ko .68 

What I should do Saturday play boll would The flower hil in the afftermoon and 
haue A chatch would the chatch on the boll lot I am going to patch on the 
manhasset team We wone the gam 16 to 4 and thun we went home 

Ro McN 3.66 

Next Saturday being that I have an holiday, I expect to go to Washington 
D.C. to spend Saturday and Sunday looking at the Pension Building, Libraies 
and the Captail etc. Into which I am very much interested in. 

I would like to see the large streets, and especially visit the cabinet the large 
room and hear the statesmen talking 



SUPPLEMENTING THE HILLEGAS SCALE 



35 



Gr Mi 3.31 

I should like to-do next Saturday to help my mother and clean all the house and 
sweep the floors and scrob. and wash windows and chop wood for her and carrie 
it in the house and obey her and my father. Go to school hight on monday morn- 
ing and to store and go out shopping with her. And if it is early mother and I 
would go to a show. Then I should help my big sister and clean house for her 
and wash windows and sweep and shake all the carpets and make the beds and 
open all the windows. Then mother and my sister in-law would give me fifty 
cent a piece. 

Je Mi 4.36 

I should like to go a way for a long journey to California. And see the nice 
fruit trees and see how large they are. It is Very nice out there. For I have friends 
out there and they say it is very nice? 

There is large trees out there that they have made tunnels through and places 
where horses can go through 

There is lots of apple, and Oranges in California They send a lot of fruit to 
New York. I an in Roslyn now and do not like it Much? I should like to stay 
there for three months but I think only can stay for a few Weeks. But all the 
same I like to go and see my friends? 

Pa Mu 2.50 

What I shall do Saturday 

I shall go out to my aunt in Linderhurst 

When I get out their, I will go out to the chickens and then see the two dogs 

When uncle otto comes home from worke Saturday night. Wee will 

get up early in the morning 

and get the chickencoop 

clean 
And in the might I will go to the move picturs with uncle otto 

Le O'B 2.40 

I am going to school tomorrow. Are you going to play with me. 

I am going to the theatre to night. Are you going to night to. I am going home 

If you will play with me I will go. Alright are you coming now to play 
Don't forget to come. " Yes, I won't forget to come. I am going to play school 
at home. I will go with you I won't with you. Are you playing with us No 
I ant playing with you. 

I Am going to play bass ball. Are you going to play to. " Yes, I am going 
to play. 

Co O'T 4.77 

Next Saturday a number of girls in my class are going on a tramping trip, 
sometimes called a Nature Study trip. As a very merry and jolly bunch of girls 
are going, I should also like to. 

And if I am not allowed to go on this Trip, I am making reckoning on spend- 
ing The day with my Aunt, For in The evening. Some friends of hers are planning 
to bring a surprise party for her. 

But as it is I really do think I would rather go on The trip, For I am sure I 
would have more fun. 

RoR .80 

I intend to go slealy rady next Saturday and mach a baby and pout a lout of 
chilon on. 

And aftir we done we will go home and mach a big fout and mach two and we 
will have a snou ball find and we will rush om the eneany and we will catep there 
fout and we will traif thin home and we will ceap ther fout fou a price and there 
will mach a nether and we will fout thin again and we will have a good time 



36 



TEACHERS COLLEGE BULLETIN 



Lu Rh 7.51 What I should ^ike to do when I leave High School 

When I am graduated from Oceanside High School, I should like to study 
Chemical Engineering in the Carnegie Institute of Technology at Pittsburgh, 
Pa. I should like to finish my work at college as soon as possible in order to begin 
the more serious business of life. This work has always held for me considerable 
fascination, and I am deeply interested in it. Material wealth and riches need 
not be the result of my choice of occupation, but rather satisfaction with my 
work, an interest in it and, above all, the realization that my life has not been 
wasted. 

Ec Si 8.01 

Saturday I should like to transport myself to one of inumerable glens dotting 
the Southern portion of our country. It would be a shady, cool retreat, where 
moss covered rocks form natural seats, — where a crystal brook would babble 
and murmur of the wood creatures, — of the plants, and of the spring, as it winds 
its way among the monarchs of the woods, — where birds would serenade me with 
their sweet, warbling melody. And there I should study the wonderful manifes- 
tations of the power of God, — the bubbles of snowy clouds floating across the 
infinite surface of the azure sky, — the animmals and their homes. And then, per- 
haps I should roam thro the leafy aisles of the wood and study the inumerable 
flowers and plants, and breathe nature-given balm — their perfume. The colorful 
pictures, — the sky with its exquisite tints, — the flowers, — the birds, — the brook, — 
would give me more pleasure and information than the greatest pictures ever 
painted. Such is my dream of a perfect day, — and to sleep beneath the gleam- 
ing stars set in the soft, dark velvet of the sky, and breathe the invigorating airs 
of the woods, — its perfect ending. 

Jo Si i.oj 

1 ever af ten noon Three olock want I go home I took my gulf club and play. And 
I had lot of fun and sum Time I play bast ball in the night. And The Saturday 
I go to the gulf, club want I go there we bast ball and. I make lot of money and 
in Sunday 

Eu Sw 7.47 My future plans 

As a general thing a boy of fifteen or sixteen has no definite plans for the 
future. His ambitions are extremely eccentric and in most cases absurd. I believe 
that one of youth's first ambitions is to become an actor. He attends some theatre 
or perhaps the circus and is deeply impressed by blase of everything. This passion 
usually last about two years. A boy's future depends upon his temperament 
and the impressions dictated by in the ordinary course of life. I am glad to say 
that I have emerged from the previously mentioned age and after careful con- 
sideration have come to the conclusion that a course at either West Point or An- 
napolis is the zenith of my ambitions. On the whole I prefer Annapolis but if_ by 
any chance I should be unable to secure an appointment there I would not object 
to West Point. Both Aacdemies offer the widest opportunities to a young man 
both from an educational and fundamental standpoint. I do not believe that a 
young man could at the present time chose a more honorable or opportune future. 

Ch Wi 416 

I am going to playball, in a corner lot near are house. I might go to Great 
Neck in the afternoon if it does not rain. I would like to go on a trip to New 
York. I think I can go. I would like to go in the woods and pick May flowers 
and Violets if they are out. I should like to go and see a ballgame, and a good 
one too. I might can go to the railroad station and see if it is a package there. 
I would like to see the Great Neck Scouts play ball tomorrow. 



SUPPLEMENTING THE HILLEGAS SCALE 



Additional Data on Standard Scores 



37 



The younger pupils in each grade will usually write better 
compositions than the older pupils, because they have somewhat 
greater academic ability than the older pupils. All results from 
academic or intelligence tests should therefore be tabulated by 
age within each grade} If a class happens to be composed of 
bright young children, the tentative standards given in the original 
discussion (page 19) will be too low; while if the class is com- 
posed of retarded older pupils, those standards will be found 
too high. The writer recommends, therefore, that only the 
normal-age group in each grade be used for comparing one school 
with another. The six-year-old pupils are to be considered 
normal for the first grade, the seven-year-olds for the second 
grade, and so on through all the grades. As a matter of actual 
fact it often takes more ability than an average pupil possesses 
to be in the eighth grade of the usual school at age fourteen, but 
theoretically fourteen-year-olds are of normal age in the eighth 
grade. 

The statement of median scores given below includes, there- 
fore, not merely the tentative standards proposed by the writer 
five years ago, but also the average of the actual qualities of all 
compositions measured in each grade, and the median achieve- 
ments of pupils of normal age in each grade. Older pupils should 
not be expected to do as well as, and younger pupils should 
be expected to do better than, the normal-age group in each 
grade. 

Median Achievement in English Composition by Grades 

Grade IV V VI VII VIII IX X XI XII 

Standard (Tentative) 3.5 4.o 4-5 5-° 5-S 6.0 6.5 6.9 7.2 

Actual Average 3-° 3-6 42 4-7 5-3 5-2 5-9 6.3 6.7 

Normal Age Pupils 3.0 3.6 4.1 4.6 5.1 5-5 5-9 6.3 6.7 

The above scores represent the work of each grade at approxi- 
mately the middle of the school year. Some allowance would 

'For evidence on this point see the writer's report on tests given in St. Paul 
(Minn.) School Survey and in the Baltimore (Md.) School Survey. 



38 



TEACHERS COLLEGE BULLETIN 



need to be made in each grade for results taken at the beginning 
or at the end of the year. 



Sample 

Ri Be 


Value 
4,0? 


ChCo 

Ha Da 


5 

2 


76 

5/| 


To Dr 


4 


86 


ChDr 

ChFr . .. 


4 

c 


98 
89 
67 


QuFr 


2 



Key to Compositions on Pages 33 to 36 

Value 
7.68 



Sample 

Ad Ja 

FrKo 68 

Ro Mc 3.66 

GrMi 3.31 

JeMi 4.36 

Pa Mu 2. 50 

Le O'B 2.40 



Sample Value 

CoO'T 4.77 

RoR 80 

Lu Rh 7.51 

He Si 8.01 

Jo Si 1.03 

Eu Sw 7.47 

Ch Wi 4. 16 



LIBRARY OF CONGRESS 



019 840 738 



